<html>
<head><meta charset="utf-8"><title>Size of the ui test suite · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html">Size of the ui test suite</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="212240424"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240424" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240424">(Oct 04 2020 at 20:33)</a>:</h4>
<p>Is the size of the ui test suite a concern, and should we make efforts to combine multiple tests into one if possible? That's what I am doing but the discussion came up in <a href="https://github.com/rust-lang/rust/pull/77512">https://github.com/rust-lang/rust/pull/77512</a> and <span class="user-mention" data-user-id="118594">@ecstatic-morse</span> disagrees, so I wonder what the compiler team thinks about this.</p>



<a name="212240568"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240568" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240568">(Oct 04 2020 at 20:36)</a>:</h4>
<p>By the way, are they run in parallel? They seem to go sequentially, but it seems it would be easy to parallelize...</p>



<a name="212240614"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240614" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240614">(Oct 04 2020 at 20:36)</a>:</h4>
<p>they run in parallel</p>



<a name="212240733"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240733" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240733">(Oct 04 2020 at 20:39)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> if you want to make the test suite run faster, print the amount of time it takes to run each test, sort it and start from the top. Trying to get PR authors to write a strictly minimal number of tests given the current structure of the code is not a productive use of time IMO. I'd rather have too much coverage than not enough.</p>



<a name="212240814"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240814" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240814">(Oct 04 2020 at 20:41)</a>:</h4>
<p>There's also been an effort to port some UI tests to standard library unit tests, which doesn't reduce coverage at all. I think this is noble as well.</p>



<a name="212240819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240819">(Oct 04 2020 at 20:41)</a>:</h4>
<p>there's a non-zero constant overhead per test, so just reducing the number of them IMO is a fair strategy to avoid test suite time blowup</p>



<a name="212240827"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240827" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240827">(Oct 04 2020 at 20:41)</a>:</h4>
<p>That said, I am happy to follow t-compiler policy here, I just don't know their policy :)</p>



<a name="212240852"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212240852" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212240852">(Oct 04 2020 at 20:42)</a>:</h4>
<p>I do not think there is policy.</p>
<p>I think it makes sense to not duplicate tests, but it can be hard to say whether we're covering the same surface area with UI tests</p>



<a name="212241334"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212241334" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212241334">(Oct 04 2020 at 20:53)</a>:</h4>
<p>maybe it would make sense to add unit tests for the compiler? Then the tests wouldn't take as long to run and you could write more of them</p>



<a name="212241377"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212241377" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212241377">(Oct 04 2020 at 20:54)</a>:</h4>
<p>ideally it would even let you run tests without having to compile all crates, you'd only have to compile up to the one you're testing</p>



<a name="212241386"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212241386" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212241386">(Oct 04 2020 at 20:54)</a>:</h4>
<p>definitely a longer-term project though</p>



<a name="212241388"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212241388" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212241388">(Oct 04 2020 at 20:54)</a>:</h4>
<p>I think that's a bit "draw the whole owl in step two" in the sense that it's not something we can "just do"</p>



<a name="212241455"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212241455" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212241455">(Oct 04 2020 at 20:56)</a>:</h4>
<p>I guess the hard part would be separating out things that can be tested on their own</p>



<a name="212242166"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212242166" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mati865 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212242166">(Oct 04 2020 at 21:16)</a>:</h4>
<p>I'd be lovely to have code coverage working and move all duplicated tests to separate repo/directory. They could be ran (bi-)weekly as a separate job just to make sure nothing regress.<br>
So we would test most important things every PR and do big test from time to time.</p>



<a name="212242258"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212242258" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mati865 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212242258">(Oct 04 2020 at 21:19)</a>:</h4>
<p>In regard to number of tests more small tests is definitely easier to work with for the developer but on the other hand it slows down CI much more than fewer big tests. Especially Windows based (who would have guessed...) runners which are already slowest runners take biggest hit from it.</p>



<a name="212251702"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/Size%20of%20the%20ui%20test%20suite/near/212251702" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Noah Lev <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite.html#212251702">(Oct 05 2020 at 01:54)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232545">Joshua Nelson</span> <a href="#narrow/stream/131828-t-compiler/topic/Size.20of.20the.20ui.20test.20suite/near/212241334">said</a>:</p>
<blockquote>
<p>maybe it would make sense to add unit tests for the compiler? Then the tests wouldn't take as long to run and you could write more of them</p>
</blockquote>
<p>Those might be a pain to write since you have to setup all kinds of state in the compiler properly :/</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>